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Abstract 

In many large scale distributed systems and on the web, agents need 
to interact with other unknown agents to carry out some tasks or trans- 
actions. The ability to reason about and assess the potential risks in 
carrying out such transactions is essential for providing a safe and reliable 
environment. A traditional approach to reason about the trustworthi- 
ness of a transaction is to determine the trustworthiness of the specific 
agent involved, derived from the history of its behavior. As a departure 
from such traditional trust models, we propose a generic, machine learn- 
ing approach based trust framework where an agent uses its own previous 
transactions (with other agents) to build a knowledge base, and utilize 
this to assess the trustworthiness of a transaction based on associated 
features, which are capable of distinguishing successful transactions from 
unsuccessful ones. These features are harnessed using appropriate ma- 
chine learning algorithms to extract relationships between the potential 
transaction and previous transactions. The trace driven experiments us- 
ing real auction dataset show that this approach provides good accuracy 
and is highly efficient compared to other trust mechanisms, especially 
when historical information of the specific agent is rare, incomplete or 
inaccurate. 

Keywords: trust management, machine learning, features, large-scale 
systems 
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1 Introduction 



Trust is an important ingredient facilitating reliable interaction^] among au- 
tonomous participants without global coordination in diverse large-scale sys- 
tems including e-commence, distributed and peer-to-peer systems, multi-agent 
systems and dynamic collaborative systems. Due to the large scale and open- 
ness of these systems, an agent is often required to interact with other agents 
with which it has few or no shared past interactions. To assess the risk of such 
interactions and to determine whether an unknown agent is trustworthy, an 
efficient trust mechanism is necessary [HI [10] . 

Traditional approaches, while effective when the necessary information is 
available, often rely upon knowledge that may not actually be available locally 
to the assessor. For instance, the interaction partner recently joins the system 
so its past behavior is quite limited; or it is difficult to form the trust path to 
derive trustworthiness of the target agent (i.e., opinions from other agents who 
have interacted with the target agent are insufficient or ingenuine). Hence, we 
want to develop a new trust mechanism to compute trust even in absence of his- 
torical information of the specific agent in question. In this paper, we propose 
a generic trust framework, which investigates trustor's local knowledge using 
machine learning algorithms. Machine learning aims to automatically learn to 
recognize complex patterns and make intelligent decisions based on existing 
datasets. Since each agent may have past transactions with other agentfl we 
argue that by investigating useful features that are capable of distinguishing 
successful transactions from unsuccessful ones, we can apply sophisticated ma- 
chine learning algorithms to analyze past transactions to learn what a successful 
transaction is and then apply the result of learning to predict trustworthiness of 
a potential transaction. This new mechanism, given its use of different kind of 
information, is meant to complement traditional models, and given its reliance 
of different set of information, should and can not be compared beyond the 
purpose of validation. 

To explain how our approach works and differs from traditional trust mech- 
anisms, let us use an analogy with our behavior in the real world. When a 
customer, say Bob, wants to evaluate the trustworthiness of a provider, say 
Sally's restaurant (i.e., to guess how he will be satisfied by Sally's food), it is 
possible to consider two different cases: 

• Bob has access to some feedbacks specifically about Sally's restaurant. 
For instance, Bob has already eaten there in the past, or maybe one of 
Bob's friends did, and gave Bob some feedbacks about it. Then Bob uses 
these feedbacks to decide whether he will eat at Sally's restaurant or not. 
This is the case that traditional recommendation systems rely upon. More 
generally these recommendation systems will try to find feedbacks about 

1 Throughout this paper, we use the terms interaction and transaction interchangeably. 

2 One weak assumption of this work is that trustor has sufficient past transactions such 
that machine learning algorithms can be performed. In case trustor is new or inactive in the 
system, i.e., no or few past transactions are available, we use a overlay network where peers 
share their local knowledge to augment the trustor's sparse knowledge. 
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Sally's restaurant among Bob's friends, or among the friends of Bob's 
friends (FoF), or from online restaurant review sites and so on. As we 
will see in the related works section, this method performs well, but has 
several drawbacks: (i) Bob has to find somehow a chain of friends that 
will provide him feedback, (ii) Bob needs to trust, and to have the same 
food taste as his feedback providers. 

• In all other cases, for example when Bob is abroad or when Sally's restau- 
rant just opened, Bob uses a different mechanism. Indeed, Bob will rely 
on his own experience about restaurants to make a first estimate on Sally's 
offer: he will look at the global presentation of Sally and her restaurant, 
the amount of other clients, the menu, the prices and the location. Then 
he will compare this with what he knows about the other restaurants 
in the same city/country, of the same type, or where he previously had 
good/bad experience with the food he wants to order. This is essentially 
how our proposal works. Instead of relying on a chain of recommendation 
to assess trustworthiness, it relies on the user's (Bob) personal experience. 
Of course, the success of this assessment relies on two hypotheses: (i) the 
user has enough experience, and (ii) it is possible to learn and predict, 
from these experiences, the outcome of a new transaction. Our experi- 
mental results, conducted using data from an auction site show that these 
assumptions do hold in (some) real scenarios, where our proposal manages 
to predict efficiently auction frauds. 

Note that our proposal is a generic trust framework, so various machine 
learning algorithms can be integrated, demonstrating that trustworthiness can 
be efficiently learned. In this work, we use two common but effective machine 
learning algorithms: linear discriminant analysis (LDA) [THIH] and decision tree 
(DT) [22j [20] as the case studies for presentation, experiments and validation 
of our proposal. LDA is a well known method for dimensionality reduction and 
classification. It takes as input a set of events belonging to k (> 2) different 
classes and characterized by various features, and finds a combination of the 
features (a classifier) that separates these k classes of events. As an example, 
linear discriminant analysis was used to differentiate subspecies of beetles based 
on measurements of their physical characteristics. DT is a widely used and 
practical method for classification and prediction by providing a classifier in the 
form of a tree structure. Its popularity is due to the ability of generating rules 
which can be easily expressed visually and in human language. 

Although LDA and DT are simple, as seen in our real auction data driven 
experiments, they are both effective and efficient. Given proposed generic and 
extendable framework, more sophisticated methods (e.g., multiple discriminant 
analysis) may be applied. The associated trade-offs are not explored in this 
paper. 

In our proposal, the agent's local knowledge (i.e., the past interactions) are 
described by a set of features. Without loss of generality, we assume two classes: 
successful and unsuccessful interactions. Note that all the interactions have the 
same feature set. For LDA, features must be quantitative, but can take values on 
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an unbound domain. The agent who needs to evaluate the potential interaction 
divides its historical interactions into two groups: successful and unsuccessful. 
He then performs a LDA on this two groups to obtain a linear classifier that 
allows him to estimate whether the potential interaction is likely to get classified 
in the successful group or not. For DT, the features can be quantitative or 
qualitative. A decision tree is firstly constructed from the training data. Then 
the algorithm classifies an example data by starting from the root of the tree 
and moving (down) until a leaf node, which is actually the classification of 
the example data. Since different machine learning algorithms may produce 
different results in the same scenario (due to their suitability) , we also investigate 
confidence of each algorithm recommendation to help trustor make the wiser 
decision. Please note that our proposal is designed for decentralized systems, 
since each agent uses its own local knowledge to evaluate trustworthiness of 
a transaction he might get involved in. However, this framework can be also 
applied in a centralized context, where all agents' local knowledge is gathered in 
a system-wide knowledge repository. In this context, multiple local knowledge 
can be aggregated to issue a better trust assessment. 

In case trustor does not have sufficient local knowledge, we propose to con- 
struct a local knowledge sharing overlay network (LKSON), which allows local 
information (i.e., intermediate results of machine learning algorithms) to be 
exchanged between trustworthy agents. Such mechanism helps inexperienced 
agents to bootstrap in the system. 

The contribution of this paper is to define a generic trust framework based 
on machine learning that is designed/suited for large-scale open systems. Since 
our proposal generally relies on machine learning algorithms (e.g., LDA and 
DT in this paper), we argue that trustworthiness of a potential transaction can 
be efficiently learned using trustor's local knowledge and the result of learn- 
ing provides a good guidance on decision making. Moreover, confidence of the 
algorithm recommendations is studied to make decision wisely. Our proposal 
can be treated as a complimentary and alternative mechanism to provide pre- 
diction for a potential interaction, especially when there is no or few (reliable) 
global information available. Moreover, compared to existing reputation based 
trust models, our proposal is more robust: (1) it mainly relies on trustor's local 
knowledge thus lowering the risk of suffering inaccurate third party informa- 
tion; (2) features are difficult to fake since malicious agent is unaware of (or 
much efforts are needed to investigate) trustor's local knowledge thus may find 
it difficult to suitably modify features to mislead trustor. Our proposal also 
allows efficient information sharing to bootstrap inexperienced agents. Simula- 
tion based evaluation shows that compared to other trust models, our proposal 
is efficient, especially when third party information is not reliable. Moreover, 
performance of our proposal is quite stable because it only relies on trustor's 
local knowledge. Real auction dataset based evaluation also demonstrates the 
efficacy of our proposal in a realistic setting, e.g., for detecting auction frauds. 

Note that since our proposal is particularly developed for evaluating trust- 
worthiness of the transactions which are conducted by unknown service providers, 
we do not discuss how to maintain trust ratings of known agents, which can be 
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easily handled by any traditional trust management mechanism. 

The rest of this paper is organized as follows: Generic trust framework 
is introduced in Section [2J In Section we demonstrate how our proposal 
works using machine learning algorithms LDA and DT as the case studies in 
subsections l3.1l and l3.2l respectivelv. Confidence of algorithm recommendation is 
studied in Section^ We then discuss the issue of sharing local knowledge among 
the agents in Section [5l Evaluation is performed on our proposal in Section [6] 
using both real online auction dataset and synthetic dataset. In Section [7] we 
present related works regarding trust management. Finally we conclude our 
work and suggest some future research directions in Section [3] 

2 A generic trust framework 

Unlike existing trust mechanisms, our proposal does not capture the behavior 
of individual agent to decide whether or not to interact with it but focuses on 
a potential interaction itself by estimating its reliability based on features of 
this interaction and trustor's local knowledge. This avoids the risk of suffering 
inaccurate third party information. 

2.1 Notation 

We refer to a participant in the system as an agent. We denote by A the set 
of all agents in the system. There are two types of agents, customers, which 
request service from other agents and providers, which provide service. An agent 
can be both customer and service provider but in a specific transaction, it plays 
only one role. A transaction in the system happens when an agent accepts 
another agent's service. To indicate the quality of a service, an agent can rate 
the transaction. In our model, the rating of a transaction is binary: successful 
or unsuccessful. O ax ,a H denotes the transaction between provider agent a y and 
customer agent a x . Note that without loss of generality, each represents a 
unique transaction. If multiple transactions happen between the same pair of 
agents, we simply create a new virtual agent with the same characteristics. Each 
transaction 6 is described by a set of features, denoted by Fq = {/q, /J,, /q}. 
Each agent a x in the system maintains a transaction database Tr a = 
& x ,ai s ®a x ,a2 > •■•}? which records historical transactions of this agent with 
other agents. Such local knowledge is used by our trust framework to perform 
machine learning algorithm to predict trustworthiness of a potential transaction. 

2.2 Feature collection 

Features can be collected from profile of the transaction partner, or the context 
of the transaction depending on specific applications. Let us take online auction 
as an example to demonstrate how features are collected. Buyer Bob is evalu- 
ating a potential transaction of purchasing a camera from the seller Sally. The 
features associated with this transaction can be collected from three aspects: 
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(1) about Sally herself, e.g., age, gender, is she from the same country as bob? 
physical distance to bob, etc. 

(2) about Sally as a user in the system, e.g., age in the system, numbers of 
successful and unsuccessful transactions, does Sally has provided a phone 
number? how complete is her profile? number of items Sally already sold, 
average delivering time, average time between end of auction and user 
comment, number of friends Sally has, etc. 

(3) the cameras Sally sells, e.g., average item price (items in the same cate- 
gory), number of the same items in stock, number of comments on these 
cameras, number of different buyers that already placed bid on it, average 
age in the system of buyers that already placed bid, etc. 

Two points regarding feature collection need to be noted: (i) Sally can cheat 
and set fake information about herself, but faking information from the 2nd and 
3rd aforementioned categories is harder since this information is maintained by 
the service exchange platform, (ii) Although, if available and appropriate, our 
proposal also uses historical information (e.g., numbers of successful and unsuc- 
cessful transactions, etc.) like traditional trust mechanisms, the information is 
used in a quite different way, i.e., it is not directly used to derive trust but is 
used as a feature to help extract relationship between the potential transaction 
and past successful/unsuccessful transactions. We will show in evaluation that 
by suitably using historical information as the features, our proposal outper- 
forms traditional trust mechanisms that directly use historical information to 
derive trust. 

We argue that compared to existing reputation based trust models, the fea- 
ture collection process of our proposal is more robust against malicious agents: 
for a traditional reputation based trust model, when trustor requests opinions 
about the target agent, malicious entities can easily counterfeit the feedbacks to 
promote or bad-mouth the target agent and trustor needs much efforts to filter 
out these false feedbacks, but for our proposal, since attackers are unaware of 
trustor's local knowledge, it is difficult to fake features associated with the po- 
tential transaction to mislead the trustor. Our proposal is thus more resilient 
to the attacks that are common to current reputation based trust systems. 

2.3 Architecture 

Fig. Q] depicts the architecture of our proposal. The framework consists of 
three components: storage component (SC), trust calculation engine (TCE) 
and knowledge collector (KC). 

Storage component is responsible for managing local knowledge, including 
agent's past transactions as well as third party knowledge collected from other 
agents. After each transaction, agent records associated information such as 
characteristics of the transaction partner, outcome of the transaction and other 
related features (done by feature extraction engine), depending on the types of 
applications. For instance, in online auction site, such features may be price or 
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Figure 1: The architecture. 



Table 1: Structure of local knowledge repository 
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category of the item, or number of items already sold by the seller, etc. (see 
Table Q] as an example using Allegro [3] dataset). These past transactions are 
used as the knowledge base to perform machine learning algorithms. 

Our trust framework is generic in two senses, the approach can be applied 
for different applications and correspondingly diverse features; and diverse ma- 
chine learning approaches may be plugged in in the framework. Trust calcula- 
tion engine is responsible for applying machine learning algorithms to predict 
trustworthiness of the potential transaction using local knowledge or collected 
knowledge as the training set. Depending on application scenario, TCE is able 
to choose the appropriate machine learning algorithm to conduct trust calcu- 
lation. For instance, if features are quantitative, discriminant analysis will be 
used, otherwise, if several features are qualitative, DT is more suitable. Actu- 
ally, subject to time/computation restriction, TCE will apply as many suitable 
algorithms as possible. In case the result is not consistent across all algorithms, 
TCE returns the result that is calculated by the algorithm which has the highest 
confidence. Measurement of such confidence is algorithm dependent and we will 
discuss this issue in Section |U 
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With sufficient local knowledge and suitable machine learning algorithms, 
trustor is able reliably predict trustworthiness of a potential transaction. How- 
ever, when local knowledge is insufficient, machine learning algorithms will per- 
form poorly. To address this issue we propose to construct a local knowledge 
sharing overlay network (LKSON) where agents are able to share their local in- 
formation (done by knowledge collector (KC)). Different from traditional trust 
mechanisms where feedbacks of the specific agent are shared, in our approach, 
agents only exchange intermediate machine learning algorithm result (see Sec. 
[5]). Such a strategy has several advantages: (1) the shared information is just in- 
termediate result of an algorithm so it is not easy to dig out agent's privacy, i.e., 
identification; (2) since information provider does not know whom the trustor is 
evaluating as well as trustor's local knowledge, it is difficult to send fake infor- 
mation to promote or bad-mouth some specific agent; (3) a lot of computation 
duplication is avoided. We will present the local knowledge sharing overlay net- 
work in detail in Section [5] We next present details of our machine learning 
based trust assessment algorithms. 

3 Machine learning algorithms for trust assess- 
ment 

A machine learning algorithm for trust assessment takes as input a set of 
trustor's past transactions with other agents and the feature vector associated 
with each transaction. The output of the algorithm is a prediction indicating 
the potential transaction is reliable or risky, with a recommendation confidence 
(see local knowledge repository (with feature extraction engine) and trust cal- 
culation engine in Fig. [T]). 

We next use two common but effective machine learning algorithms, i.e., 
linear discriminant analysis and decision tree as the case studies to demonstrate 
how our proposal works. Algorithm recommendation confidence is discussed in 
Section |U 

3.1 Case study: Using linear discriminant analysis 

Consider a scenario where a customer a x encounters a potential service provider 
a y and a x has no prior experience with a y . We assume that a x can obtain fea- 
tures of this potential transaction 6 ax ,a (e.g., from a y 's personal profiles and 
from the transaction offer it received). The set of such features of Oa x ,a B is de- 
noted by Fe ax , ay = ife a a > f® a a > — >/e a a }• ( to simplify the notation, we 
use /* instead of fX when the context is clear). So the potential transaction 

is represented by vector p — (f 1 f 2 f 3 . . . f d ). 

We assume that a x has recorded n historical transactions. To estimate reli- 
ability of the potential transaction, a x classifies its historical transactions into 
two disjoint groups, the successful transaction group G s and the unsuccessful 
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transaction group G u according to the outcomes of these transactions. The two 
transaction groups are represented a§ 
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n s and n u are the sizes of successful transaction group and unsuccessful 
transaction group respectively. Please note that n = n s + n u . 

a x performs linear discriminant analysis to classify the potential transaction 
as belonging to successful or unsuccessful transaction group to decide whether 
or not to transact with the corresponding service provider for the specific trans- 
action. The mathematical operations are the following. Let h x be a x x 1 
(column) vector of ones. 

a x first calculates centroid of each group. That is, for each feature, we 
calculate average value of this feature across all transactions in the group. 



• h£ G s and c v 



h T C 



(3) 



Similarly, the global centroid is calculated by averaging each feature across 
all past transactions: 



1 
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(4) 



In LDA, the internal variance (within-class scatter matrix) and external 
variance (between-class scatter matrix) are used to indicate the degree of class 
separability, i.e., to what extent are the successful transactions distinguished 
from the unsuccessful transactions. The internal variance, which is the expected 
covariance of each group is given by Eq. [5] and [6] 
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(6) 



h n3 and h nu are used to center the values around the centroids c s and c u . So 
the overall within-class scatter matrix is calculated as the weighted sum of each 



3 We assume an agent is aware of context of the transaction. For instance, when evaluating 
an unknown book seller, the buyer only considers past transactions regarding purchasing 
books, not that regarding purchasing other irrelevant items like digital cameras. 
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group's internal variance, where the weight is fraction of transactions regarding 
the corresponding group: 



S w = ~(n s SZ,+n u S%). (7) 

Then a x calculates external variance, which is actually the covariance of the 
two groups, each of which is represented by its mean vector (Eq. [8]). 

S b = ~(n s (c s - c) T (c s - c) + n u (c u - c) T {c u - c)), (8) 

The mixture scatter matrix, i.e., the total variance of the system is obtained 
by summing up internal variance S w and external variance Sb (Eq. [5]). 

S m = S w + Sb- (9) 

LDA aims to find a projection direction (a transformation) v that maximizes 
the inter class variance and minimizes the intra class variance. Formally, the 
criterion function (see Eq. 1 10[) is to be maximized. 

v T S b v 

J[v) = Ws~^ ( 0) 

The projection direction v is found as the eigenvector associated with the 
largest eigenvalue of 5~ 1 5ft. We then transform the two groups of transactions 
using v. Similarly, the potential transaction p — (f 1 f 2 f 3 ... f d ) is also trans- 
formed and classified by measuring the distances between transformed potential 
transaction and the two groups (i.e., centroid). Eq. [TTl[T2"l show how to calculate 
such distances: 

D s =v T p-v T c s (11) 



D u = v T p - v T c u (12) 

If D u > D s , then transaction p is predicted as successful, otherwise it is 
predicted as unsuccessful. 

Note that we try to collect as many features of a transaction as possible, 
and the algorithm filters out the not-so-relevant variables for us. That is to say, 
the feature which is more capable of distinguishing successful transactions from 
unsuccessful ones will have more impact on the final classification result. 



3.2 Case study: Using decision tree 

The basic idea of decision tree is to classify objects by sorting them from root 
of the tree to some leaf node, which provides classifications of the objects. Each 
node (except the leaf nodes) in the tree represents some feature of the object, 
and the value of such feature is the criteria by which the current set of objects 
are put in the final classifications or further split into smaller classes that will be 
classified according to other selected features (i.e., child nodes of current node). 
This process continues until all objects are classified. 
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When encountering a potential transaction from an unknown agent, trustor 
a x has n past transactions with other agents, denoted by Tr ax = {Oi, 82, Q n }- 
Each past transaction is described by the same set of features Fq = {/q, /@, / 
Such knowledge is used to construct the decision tree to generate a classifier 
(rule) to predict whether a potential transaction is risky or nolQ. 

From a given past transaction set, it is possible to construct several different 
decision trees, depending on the order in which transaction features are tested. 
Since building an optimal decision tree (e.g., optimal with respect to its size) 
is NP hard [TT] and therefore, several heuristics have been proposed to build 
efficient learning trees (e.g., ID3 [55], C4.5 [23], etc.). Without loss of generality, 
we hereafter present one of these heuristics, namely ID3 [52]. ID3 is a well known 
algorithm that relies on information gain 18j to select the classifying features 
at each node of the tree. Information gain is measured by entropy. 

Given the binary categorization (i.e., successful and unsuccessful) and past 
transactions, we denote the proportion of successful and unsuccessful transac- 
tions by p s and p u respectively. Then the entropy of all past transactions is: 

Entropy(Tr aic ) = ~p s log 2 (p s ) - pJog 2 (p u ) (13) 

Entropy is used to characterize (im)purity of a collection of examples. From 
Eq. [T2]we can see entropy will have the minimum value of when all past trans- 
actions belong to one class (i.e., successful or unsuccessful) and the maximum 
value of log 2 2 = 1 when past transactions are evenly distributed across the two 
classes. Using entropy, we now calculate information gain of every feature (with 
respect to a set of past transactions Tr) to determine the best feature to choose 
for a node in the decision tree. For each feature fi, we assume it has a set of 
values (e.g., discrete variable) or intervals (e.g., continuous variable), which is 
denoted by V(fi). For each v £ V(fi), we denote the set of past transactions 
that are associated with v for feature fi by Tr v . The information gain of feature 
fi is thus calculated by: 

XV J 

IGainCTr, fi) = Entropy (Tr) — } - — -p Entropy (Tr v ) (14) 

\Tr\ 

The information gain of a feature measures expected reduction in entropy 
by considering this feature. Clearly, the higher the information gain, the lower 
the corresponding entropy becomes and thus the better the classification of past 
transactions is achieved (by considering the corresponding feature). 

Using information gain, a decision tree is iteratively constructed by maxi- 
mizing information gain at each step. We next describe how decision tree is 
constructed (Algo. HJ by ID3 algorithm. 

The ID3 algorithm input is trustor a x 's past transactions Tr am and the fea- 
ture vector F that describe each transaction. If all transactions are successful or 
unsuccessful, the algorithm simply returns a single node (i.e., root) tree, labeled 
as '+' or '-' respectively (Line 2-7). If feature vector is empty, the algorithm 

4 Remind that we assume binary result of a transaction. 
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Algorithm 1 Decision tree construction (code for trustor a x ). 
1: Trust JD3{Tr ax ,F) 

2: if All past transactions are successful then 

3: Return single node tree Root, with label = +. 

4: end if 

5: if All past transactions are unsuccessful then 
6: Return single node tree Root, with label = -. 
7: end if 
8: if F = then 

9: Return single node tree Root, with label = outcome of the transactions 

that dominate all past transactions. 
10: end if 

11: Calculate information gain of all features and choose one feature (say, /), 

which has the highest information gain as the Root node. 
12: for each value v of / do 
13: Add a new branch below Root. 

14: Let Tr 0x f v be the set of past transactions that have value v for feature 
/• 

15: if Tr ax j, v = then 

16: Below this branch add a leaf node with label = outcome of the trans- 
actions that dominate all past transactions. 
17: else 

18: Below this branch add a sub-tree: 

19: Trust_ID3(Tr Qxi/ ,„,F - {/}). 

20: end if 

21: end for 

22: Return Root. 



returns a single node tree, labeled as '+' if most of past transactions are success- 
ful or '-' otherwise (Line 8-10). Then the algorithm calculates information gain 
of each feature using Eq. [14] The feature (say, /) with the highest information 
is selected as the root of the tree. Next, new branches are added to the root 
node according to possible values of /. For each possible value v, we denote 
the set of past transactions that have v for / by Tr amt f jV . If Tr amJ , v = 0, the 
algorithm adds a leaf node to the corresponding branch with label = '+' or '-' if 
most of past transactions are successful or unsuccessful respectively. Otherwise, 
the algorithm recursively executes itself with 2V * v and F — {/} as the input 
parameters (Line 12-21). The algorithm terminates either when all the features 
have been tested or the tree perfectly classifies the past transactions. 

When encountering a potential transaction, a x tries to classify it as a risky 
transaction or not by comparing the corresponding features from root of the 
tree down to a certain leaf node. Note that for simplicity, we do not discuss 
other issues in decision tree such as avoiding overfitting the data, etc. 
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4 Algorithm recommendation confidence 



Given a features vector, a set of past transactions and appropriate machine 
learning algorithm, our trust framework is able to return a recommendation of 
the potential transaction: be it risky or not. Confidence of such recommendation 
relies on characteristics of input (e.g., discriminating power of the features, 
volume of past transactions) and performance of the specific machine learning 
algorithms in us^B So by performing the same algorithm, trustors with different 
local knowledge and/or feature vectors may have quite different confidence on 
the recommendations. We denote a £ [0, 1] as the algorithm recommendation 
confidence, where represents the algorithm recommendation is not confident at 
all (i.e., the recommendation is equally random) and 1 represents the algorithm 
recommendation is completely confident. Algorithm input is denoted by < 
F, Tr >, where F = {fi, /2, •■•} represents the feature vector and Tr = Tr s UTr u 
represents the set of past transactions: Tr s = {6f, 0|, ...} and Tr u = {61,62, ■■■} 
are the sets of successful transactions and unsuccessful transactions respectively. 
We next present that given different inputs, how to measure confidence of the 
algorithm recommendations still using linear discriminant analysis and decision 
tree as the demonstration examples. 



4.1 Measuring recommendation confidence for LDA based 
algorithm 

According to theory of linear discriminant analysis based algorithm (refer to Sec- 
tion GO}, each transaction is described by the input feature vector F (i.e., each 
transaction is represented as a point in |i^|-dimensional space.). So confidence 
of the algorithm recommendation greatly depends on discriminating power of 
the input features. If all the features are discriminating enough, LDA is able to 
separate the successful and unsuccessful transactions clearly and minimize the 
variance of each transaction group. Otherwise, the two transaction groups Tr s 
and Tr u may overlap, thus making classification of the potential transaction 
less confident. In other words, algorithm input determines the LDA projection 
direction. After having the project direction, transaction group centroids are 
transformed to lower dimension space. The potential transaction (after trans- 
formed) is classified as risky or not by measuring the distances between it and 
the two transaction group centroids. We denote the distances between the po- 
tential transaction and successful/unsuccessful transaction group by D s /D u . 
Note that each set of distances correspond to one specific input: < D S ,D U > 
= LDA(< F,Tr >), where LDA() performs linear discriminant analysis to gen- 
erate distances using the given input. The confidence of the algorithm recom- 
mendation is then calculated as: 

\Ds-Du\ 

aLDA = ^TaT (15) 



5 Given the same input, different machine learning algorithms may generate different rec- 
ommendations (perhaps due to their suitability in different application scenarios). 
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It is clear from Eq. [13] that when D s = D u , <tlda = 0, i.e., the potential 
transaction is classified as risky or not with the same probability. When D s 
or D u is 0, cjlda — 1, i-e., the potential transaction is recommended to be 
safe or not confidently. With old A, trustor is able to measure how confident 
the algorithm outcome is and personally decides whether or not to trust the 
algorithm recommendation. 

4.2 Measuring recommendation confidence for DT based 
algorithm 

Since decision tree is constructed according to information gain of each fea- 
ture (refer to Section I3.2[) , it is natural to measure algorithm recommendation 
confidence using feature's information gain. According to information theory, 
information gain of a feature measures expected reduction in entropy by con- 
sidering this feature. If a feature is discriminating enough, the corresponding 
information gain is high (maximum value is log-il = 1), i.e., by appropriately 
choosing values of this feature, the associated successful transactions and un- 
successful transactions are separated clearly, thus the potential transaction can 
be accurately classified by this feature. Otherwise, this feature may not help 
classify the potential transaction accurately. So with different input data (i.e., 
feature vector and set of past transactions), different information gains for a 
specific feature may be generated and hence different confidence on the classifi- 
cation results may be obtained. 

For each feature /, (with respect to a set of transactions Tri), we calculate 
the corresponding entropy Entropy(Tri) and information gain IGain(Tri, fi) 
using Eq. [T3] and Q3] respectively. We then calculate the extent of entropy 
reduction by feature fc 



Ti can be interpreted as the accuracy of classification of the potential trans- 
action by feature /, (i.e., at the node corresponding fi in the decision tree.). We 
notice from Eq. Ql)] that if fi is extremely discriminating, its information gain 
is Entropy(Tri) (i.e., the second part of Eq. Q3] is 0) and thus is 1, which 
means the potential transaction classification by fi is completely accurate. If 
feature fi is incapable of discriminating, its information gain is and thus 
is 0, which means classification of the potential transaction is inaccurate at all 
(i.e., the classification is equally random.). 

After having accuracy of classification of each feature, we calculate algorithm 
recommendation confidence as: 



So, if all the features are quite discriminating, the algorithm recommendation 
confidence is approaching 1. Otherwise, any less discriminating feature shall 



IGain{Trj, fj) 
Entropy (Tn) 



(16) 




(17) 
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greatly impact the final confidence. This also follows the decision rule of the 
decision tree based algorithm. 

5 Local Knowledge Sharing Mechanism 

Our trust framework works under the assumption that the trustor has sufficient 
local knowledge such that machine learning algorithms can be performed. How- 
ever, in some cases, agents may have no or sparse local knowledge (e.g., the 
bootstrapping agents recently join the system or do not interact with others 
frequently). To make our proposal adaptive to such a scenario, we propose to 
construct a overlay network dedicated to exchange/share agents' local knowledge 
to help inexperienced agents estimate trustworthiness of a potential transaction. 

5.1 Local Knowledge Sharing Overlay Network (LKSON) 

To evaluate trustworthiness of a potential transaction, inexperienced agents have 
to request other agents' knowledge. A local knowledge sharing overlay network 
(LKSON), which is a virtual network on top of current network infrastructure 
(e.g., P2P network) is thus constructed. Fig. [5] depicts a LKSON, which is 
represented by a directed graph. We give definition of LKSON: 

Definition 1 (Local Knowledge Sharing Overlay Network) A local knowl- 
edge sharing overlay network (LKSON) is a Weighted Directed Network (WDN), 
denoted by LKSON = (V,E,T): 
V = {ai,a2,a^, ...} is the set of agents in LKSON. 

E = {ei y j\a,i G V A ° j £ V f\ a i 7^ a j} * s se t °f directed edges represent- 
ing the trust relationship in terms of providing correct local knowledge between 
knowledge requester ai and requestee aj . 

T : E — >• [0,1], trust score Uj is assigned to edge eij and expresses from view- 
point of di, how trustworthy aj is in terms of providing local knowledge. 

We set that the trust score falls into the range of [0,1], where 1 represents 
completely trustworthy and represents completely untrustworthy. Trust score 
is determined according to requestee's past performance (i.e., quality of provided 
local knowledge) . After each interaction, knowledge requester a, evaluates use- 
fulness of the information provided by the requestee aj and accordingly updates 
trust score of aj. Please note that since agent may have very personalized per- 
spective on the system, even if requestee aj provides correct local knowledge, 
it may not be useful to the requester. In this situation, Oj's trust score will be 
lowered although it is honest. We will discuss details of trust score update in 
Section O 

Notice that LKSON is constructed to ensure reliable local knowledge sharing 
to help inexperienced agents estimate trustworthiness of a potential transaction, 
but is not used to discover trustworthy transaction partner (i.e., LKSON is not 
a trust overlay network like that in |33j ) because it has been reported in related 
literature [32], [28] that the agents who provide high quality service may not 
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Local Knowledge Sharing Overlay Network (LKSON) 




Underlying Network 



Figure 2: A local knowledge sharing overlay network. 

report genuine feedbacks and vice versa due to various reasons. In the case that 
the agents who provide useful information also act honestly in a transaction, 
LKSON can be used to help promote successful transactions (i.e., selecting re- 
liable agents who have high trust scores as the service providers). However, the 
goal of this work is to design mechanisms to estimate trustworthiness of a po- 
tential transaction, which is conducted by an unknown agent (i.e., no historical 
information is available), so discussion on relying LKSON to estimate agent's 
trustworthiness like traditional trust mechanisms (e.g., feedbacks aggregation 
[2], [J], [3D]) web of trust [T] [T3], etc.) is out of the scope of this paper. 

In order to collect useful information, inexperienced agents only request the 
agents who are trustworthy (i.e., trust score is high) in terms of providing useful 
local knowledge. So in our approach, each agent a' in the system maintains a 
Trusted Knowledge Provider list PR a > which stores the agents that this agent 
has trust relationship in terms of providing useful local knowledge. Initially, 
when requestees' past performance is not available, an inexperienced agent re- 
quests information by exploring the social network. For instance, it may request 
its "familiar" agents (e.g., friends or colleagues in the real world, etc.); or the 
agent may request "special" agent/entity (if any) in the network (e.g., super 
peer in hybrid peer-to-peer network, or advisor in Epinions 7 , or authorized 
third parties, e.g., "official" source in public key infrastructure, etc.). In the case 
that no "familiar" or "special" agents are there (i.e., no useful information at 
all), the inexperienced agent will choose knowledge provider randomly to create 
its Trusted Knowledge Provider list. 
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We next present detail of LKSON design using LDA as the demonstration 
example. Some issues like what kind of information is shared, how to combine 
knowledge of multiple sources and how to update relationships among agents in 
LKSON, etc. will be discussed. 

5.1.1 What is Shared? 

To help an inexperienced agent detect a potentially risky transaction, two types 
of local knowledge, which are represented as a tuple <v, (c s , c u )> are shared 
among the agentt|j: 

(i) the projection direction v that maximizes the inter class variance and 
minimizes the intra class variance (see Sec. I3.ip . 

(ii) centroid of the successful group c s and unsuccessful group c u . 

The main reasons that these two types of knowledge are shared: (1) the knowl- 
edge providers have no motivation to send fake information because they are 
unaware of requester's potential transaction partner thus are unable to de- 
fame/promote the transaction partner or mislead the requester maliciouslj0. 
That is, this is safer than directly requesting others' trusts in the target agent, 
which is very easy to be attacked). (2) the projection direction v is shared, 
which means requester does not need to compute it so a lot of computation is 
saved. The process of deriving projection direction is the most computation- 
intensive part of LDA. (3) the shared knowledge is just projection direction and 
centroid of successful and unsuccessful groups so it is not easy to dig out agent's 
privacy through shared information. This is important to knowledge providers 
and their past transaction partners. (4) The shared knowledge is of small size 
so the communication overheads incurred are marginal compared to the cost of 
sending the whole local knowledge base. 

5.1.2 Combining Third Party Knowledge 

When an agent (with no local knowledge) encounters a potential transac- 
tion p = (/ 1 / 2 / 3 -.-/ d ), where / represents feature, requests local knowl- 
edge of other agents that are recorded in a^s Trusted Knowledge Provider 
list PROVIDER(ai) to predict trustworthiness of p. We denote the set of 
knowledge providers who response and provide the requested information by 
KPi = {a\, a l 2 , a l 3 , a l n .} . So the size of KPi is n^. The corresponding trust 
scores of these knowledge providers are denoted by TS = {t\, t%, t\, t l n .}, 
where i*- represents a^s trust in the jth knowledge provider. The local knowl- 
edge shared by the jih knowledge provider is represented by <Vj, (c s j, c u j)>. 

6 We assume that all the agents have the same features such that the projection direction 
v can be shared and combined meaningfully. 

7 This reason is sometimes not true (e.g., the knowledge provider may learn requester's 
transaction partner from other ways, or the provider simply sends incorrect information for 
fun, etc.). We address this issue by updating Trusted Knowledge Provider list, which will be 
described in Sec. 15.21 
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The collected knowledge is combined to produce a classifier that helps es- 
timate whether p is likely to get classified to the successful group or not. The 
resultant projection direction is derived as the weighted sum of all collected 
projection direction: 

v i = ^2w j v j (18) 

3 

Similarly, the resultant centroid of successful group and unsuccessful group 
are calculated as: 

rii 

C s ,i = /]WjC s> j (19) 
3 

rii 

c u ,i = ^w J c u j (20) 

i 

The weight Wj for the jth knowledge provider is determined by its trust 
score: 




(21) 



After combining other knowledge providers' projection direction (t>j), 
transforms the combined centroid (c Sj i, c Mj i) using Vi. Similarly, the poten- 
tial transaction p is also transformed and classified using Euclidean distance of 
transformed data from each transformed centroid. The distances are computed 
using Eq. Eland EE] 



5.2 LKSON Update 

Since behaviors of the agents may change overtime (e.g., honest knowledge 
provider may provide fake information or they may change their perspective 
on the system thus making the shared knowledge not useful to the requester 
any more, etc.), it is essential to update trust scores of the knowledge providers 
such that the inaccurate/useless knowledge has less impact on the final decision. 

It is effective if we adapt trust scores of the knowledge providers accordingly 
every time a transaction is completed. However, such strategy is inefficient be- 
cause it incurs unnecessary computation overheads. That is, only trust scores 
of the knowledge providers whose local knowledge produces contrary prediction 
on the outcome of the transaction need to be updated. So we adapt a strat- 
egy by trading off computation overhead and performance, that is, after one 
transaction, if its outcome is unsuccessful, the requester ctj evaluates usefulness 
of local knowledge of each knowledge provider. For knowledge provider a*, 
simply uses Vj as the projection direction to calculate distances of transformed 
potential transaction from the transformed centroid (see Eq. [Til and [T2|) . If 
the transaction is predicted as successful, it means the corresponding knowl- 
edge is incorrect or useless to ai so trust score of the knowledge provider a*- 
must be lowered such that it has less impact on future decision. Otherwise, 
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the trust score should be raised. The trust score update follows a Bayesian ap- 
proach [14] , which takes binary ratings as input and is based on computing trust 
scores by statistically updating beta probability density function (PDF). The 
posteriori trust score is computed by combining a priori trust score with new 
evidence. For knowledge provider a* , we denote numbers of its successful and 
unsuccessful predictions by s* and respectively (i.e., the original trust score 

i s s \+ u i + 2 ^' ^° a i' s trust score is updated after one transaction using Eq. [22] 
(the new prediction is correct) and Eq. [231 (the new prediction is incorrect). 



t) = . 3 . (22) 

4^1 



t) = (23) 



On the other hand, if the transaction is successful, the requester will evaluate 
usefulness of a small fraction (e.g., 20%) of the third party knowledge. This can 
help (1) avoid incurring too much unnecessary computation overheads because 
if the transaction is successful, it means combination of current third party 
knowledge is accurate enough to make a good prediction so no update is needed 
urgently and (2) proactively adjust weights of the third party knowledge thus 
making the combined knowledge more accurate (for the next prediction) while 
keeping the computation overheads low. 



6 Evaluation 

6.1 Simulation settings 

We use real dataset collected from an Internet auction site Allegro [3] as well 
as synthetic data to conduct experiments. The Allegro dataset contains 10,000 
sellers, 10,000 buyers, more than 200,000 transactions and over 1.7 million com- 
ments. In the experiments, a transaction is considered successful if its feedback 
is positive, otherwise, it is considered unsuccessful. We extract three features 
from Allegro data, i.e., Fx: category of the item; F^. price of the item, F3: 
number of items already sold by the seller when the transaction occurs and F4.: 
fraction of non-positive feedbacks of the seller. We evaluate performance of our 
proposal, i.e., LDA based approach and DT based approach by studying their 
capabilities of detecting Internet auction fraud. When a buyer encounters a po- 
tential transaction, which is conducted by an unknown seller (i.e., no historical 
information of this seller is available), this buyer will gather past transactions 
regarding the item category (i.e., Fx) and then perform LDA or DT to estimate 
trustworthiness of this transaction using features F2, F3 and F4. 

Allegro has a very friendly environment: only around 0.9% of the trans- 
actions are labeled negative explicitly. To test our proposal in a more hos- 
tile environment, and also to compare proposed algorithms with other existing 
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trust mechanisms, we generate synthetic dataset consisting of 10,000 nodes. 
Each node is either good, which provides service satisfactorily or gives genuine 
feedbacks when requested, or malicious, which cheats others in transactions or 
provides false feedbacks when requested. To make the simulation environment 
more realistic, there are no completely good or bad nodes. We set good nodes 
and malicious nodes behave maliciously with probabilities of 15% and 85% re- 
spectively. We denote the fraction of malicious nodes in the system by P m . In 
the simulation, any node (trustor) may request service from other nodes (service 
provider or the target node). Once the transaction with service provider (target 
node) is done, trustor rates the transactions to indicate whether they are suc- 
cessful or not. The trustor stores information of these transactions (see Table 
[J) such as their outcomes and the associated features. In synthetic dataset, 
to construct LKSON, each node also maintains a Trusted Knowledge Provider 
list, which records other nodes that will be requested if this node does not have 
sufficient local knowledge. 

We also need to generate synthetic features. Note that each transaction is 
described by four features^. Let X s and X u be the random variables which rep- 
resent values of features of successful and unsuccessful transactions respectively. 
We assume that X s and X u follow normal distribution, i.e., X s <~ J\f(/i,a) and 
X u ~ Af(fi + 9,a). In our experiments [i = 1, a = 0.1 and 9 £ [0, 1] is the 
separability factor that we use to tune the overlap between values of features 
of successful transactions and that of unsuccessful ones. When 9 — 0, values of 
features for successful and unsuccessful transactions overlap completely, while 
when 9 is large enough, it means values of features for successful and unsuccess- 
ful transactions are separated clearly. In other words, 9 allows us to modify the 
learnability of the transaction classes. Note that since only our approaches are 
based on features, 9 only impacts performance of our approaches. 

We compare our proposal with three existing models: (1) Random Selection: 
The simplest approach is to engage or reject a potential transaction randomly. 
In the simulation, we set the probability that trustor accepts a potential trans- 
action to 50%. This model is used as the experiments benchmark. (2) Feedback 
Aggregation: In this model [21 [221 130]) if trustor does not know target node, 
it asks other nodes across the network and aggregate feedbacks to derive tar- 
get node's trust. False feedback filtering mechanism (from TRAVOS [30]) is 
applied to improve performance (3) StereoTrust [17]: Trustor forms groups of 
agents with which it has past experience according to the features and derives 
trust rating of the target agent by combining group trusts to which the tar- 
get agent belongs. Please refer to related work section for a brief summary of 
StereoTrust. 

Note that we only compare our proposal with the existing models when 
synthetic data is used because we lack the information of how users in Allegro 
apply these existing models. 

The metrics we use to evaluate performance of trust mechanisms include: 

8 We also conducted experiments using other numbers of features and obtained the similar 
qualitative trends. 
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• false positive rate 

The transaction is unsuccessful but the algorithm predicted that it would 
be successful. 

• false negative rate 

The transaction is successful but the algorithm predicted that it would be 
risky. 

• overall falseness 

Sum of false positive rate and false negative rate. 

Each experiment is repeated 10 times, and error bars are added indicate 
deviation of each running. 

6.2 Results 
6.2.1 Real dateset 

We rank the 10,000 buyers according to number of their past transactions, i.e., 
the first buyer has the most past transactions and the last one has the least 
transactions. We select subset Ub of these buyers starting from the first one. 
Each buyer evaluates 100 randomly selected transactions (50% are successful 
and 50% are unsuccessful) . We vary the size of Ub to investigate effect of local 
knowledge volume. 



Fig. 3(a) demonstrates performance of our approach (LDA and DT based), 
i.e., how average rates of false positive and false negative evolve when Ub varies 
from 5 to 500. As expected, all falseness rates increase when Ub grows. This 
shows the impact of local knowledge on our proposed algorithms: when £/& is 
small, it contains only experienced agents, that all have enough past transactions 
to allow our approach to issue accurate predictions. As Ub grows, it contains 
more and more inexperienced agents, for which the predictions are less accu- 
rate. We also notice that DT based algorithm is less accurate than LDA based 
algorithm. This is because basic decision tree is more suitable for qualitative 
features while features identified in auction data arc quantitative (i.e., it is not 
easy to obtain the appropriate thresholds to efficiently split the training data.). 

Fig. |3(b)| shows the distribution of numbers of individual buyers' past trans- 
actions (only first 3000 are shown). Note the logarithmic scale for y-axis: the 
number of past transactions is quickly decreasing. For instance, less than 100 
agents have more than 20 past transactions. Estimating the minimal number 
of transactions that allow our approach to be precise is challenging, since not 
all transactions have the same importance (e.g., two very different transactions 
will be much more useful for our approach than two identical transactions). 
However, in this set of experiments, we estimate empirically that when num- 
bers of transactions is over 6, the potential transaction can be relatively reliably 
predicted (i.e., the overall falseness rate is smaller than 0.1 for LDA based al- 
gorithm and 0.19 for DT based algorithm). Simulation results thus show that 
our approach is capable of quickly "learning" what a successful or unsuccessful 
transaction is. 



21 



0.16 

0.14 

0.12 

| 0.1 
<n 

<S 0.08 
™ 0.06 
0.04 

0.02 



■^Our approach (LDA) false positive 
S-Our approach (LDA) false negative 

— Our approach (DT) false positive 

— Our approach (DT) false negative 




150 200 250 300 350 400 
Number of buyers 



450 500 



(a) Performance of our approach with varying number of buyers. 



« 10 
E 



1000 1500 2000 

Index of buyers (after sorting) 



(b) Numbers of individual buyers' past transaction. 



Figure 3: Experiments using Allegro dataset. 



In online auction sites, most of the buyers are inexperienced (local knowl- 
edge is insufficient). So to predict trustworthiness of the potential transaction, 
the inexperienced buyer will collect other buyers' local knowledge to perform 
our approach by constructing LKSON. LKSON is initialized by connecting this 
buyer and other buyers according to categories that they are interested in (i.e., 
the item this buyer is going to buy is in the category from which the connected 
buyers used to buy items). Third party knowledge indeed helps inexperienced 
buyer predict trustworthiness of the potential transaction, however, it also in- 
troduced inaccurate information, which may influence accuracy of prediction 
because third party knowledge only reflects knowledge provider's personalized 
perspective on the system. For instance, one buyer can tolerate one day delay of 
item delivery (successful transaction) while another buyer can not (unsuccessful 
transaction). We thus need to estimate how many other buyers are requested 
to gather sufficient knowledge while trying to keep influence of inaccurate infor- 
mation as low as possible. 

Fig. 2] demonstrates how performance of our approach (LDA based) varies 
with the varying number of requested buyers (from 10 to 110 with 10 as incre- 
ment). We observe that the general trends of the false positive rate and false 
negative rate are: they decline with the increasing number of requested buyers 
and stop at a certain point, then they start ascending. This is because when 
the number of requested buyers is low, there are not sufficient past transac- 
tions available thus affecting the accuracy of prediction (i.e., not statistically 
significant). On the other hand, when the number of requested buyers is high, 
lots of transactions are collected, which means more inaccurate information is 
incorporated, hence the accuracy of prediction is also influenced. So the lowest 
point is the optimal result that our approach is able to achieve and the corre- 
sponding number of requested buyers is the design parameter we are looking 
for. From the figure we can see the most suitable number of requested buyers in 
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this scenario is 60 and the corresponding false positive rate and false negative 
rate are 0.08 and 0.042 respectively. 
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Figure 4: Performance of our approach (LDA) with LKSON (real dataset). 



6.2.2 Synthetic dataset 

We compare performance of our approach with other existing trust models (Fig. 
El El [7] and [8]). Notice that when local knowledge is sufficient (i.e., at least three 
successful transactions and three unsuccessful transactions), trustor relies on 
its local knowledge to perform our approach. Otherwise, trustor will integrate 
third party information via LKSON. 

We observe that when P m is low, feedback aggregation model performs quite 
well. This is because when most of nodes in the system are honest, the aggre- 
gated feedbacks can accurately predict behavior of the service provider thus 
keeping rates of false positive, false negative as well as overall falseness low. 
However, with the increase of P m , various falseness increases sharply. So accu- 
racy of feedback aggregation model heavily depends on percentage of malicious 
nodes and thus is not suitable for a hostile environment. 

One interesting thing about feedback aggregation model is that its false 
negative rate decreases when P m is larger than 0.5 (Fig. 5(b)| |6(b)| |7(b)| and 
|8(b)| . The possible reason is that when P m is high, the amount of false feedbacks 
increases thus increasing the falseness probability. On the other hand, the total 
amount of successful transactions decreases generally, so the false negative rate 
decreases. 

Random selection model, as we expected, provides a stable rate of overall 
falseness of 0.5 because we let trustor accept or reject transactions equiprobably. 
This model is the benchmark of the experiments, demonstrating falseness rate 
evolution in the scenario where no trust management is applied. 

StereoTrust model outperforms feedback aggregation in general especially 
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Figure 5: Comparison of our approach with other approaches (6 — 0.2). 
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(a) False positive. (b) False negative. 

Figure 6: Comparison of our approach with other approaches (0 = 0.4). 

when Pm is high. This is because even though StereoTrust also relies on some 
feedbacks to predict behavior of the target node, it only requests nodes that 
are honest (from its own perspective) thus the probability of receiving false 
feedbacks is relatively low. 

We observe that DT based approach is less accurate than LDA based ap- 
proach in all scenarios. This result conforms to the result when real data is 
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used (Fig. 3(a) ) and further proves that LDA based algorithm is more suitable 
to the scenarios where features are quantitative. So to sum up, LDA based 
approach, which has the lowest rate of various falseness outperforms all other 
models (except for the scenarios that 9 is quite small, i.e., when 9 < 0.4). When 
9 is quite small, both LDA based approach and DT based approach suffer from 
low accuracy of classification. Finding features that can distinguish successful 
transactions from unsuccessful ones is thus the key of our approach. We also 
notice that performance of our approaches are quite stable (not affected by P m 
very much). This is because our approach mainly relies on local knowledge, 
which is the most reliable information source. Note that since our approach 
relies on different kinds of information, it may not be rather fair to compare 
with other models that only use one kind of information (i.e., target agent's his- 
torical information). Moreover, in some application scenarios, the information 
used by our approach (e.g., features) may not be always available. Nevertheless, 
whenever such information is there, our approach indeed produces reasonable 
results thus demonstrating its efficiency 
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Figure 8: Comparison of our approach with other approaches (6 = 0.8). 

7 Related Work 

Most existing reputation based trust management mechanisms derive an agent's 
trustworthiness based on that specific agent's past behavior (e.g., [Tl I2TI [TBI l4l 
34. 3TJ[29]). If the assessing agent has sufficient direct experience with a provider 
agent, this provider's future behavior can be reliably predicted [21] , However, 
in large-scale distributed settings, direct experience is often unavailable. In this 
case, the assessing agent may resort to "indirect experience" - opinions about 
the assessed agent obtained from other agents [JJ [J?l [J5J [351 [3TJ . Simple aggre- 
gation (like a seller's ranking on eBay) relies on access to global information like 
history of the assessed agent's behavior. Alternatively, transitive trust models 
[U E] forms chains of trust relationships among agents. However, transitive 
trust is not always realistic in the real deployment and it has several drawbacks: 
(i) this method does not handle wrong recommendations properly, which affect 
the accuracy of derived trust seriously [6] . (ii) This method does not provide a 
mechanism for updating trust efficiently in a dynamic system, (iii) Establishing 
a trust path, even if such a path exists, is nontrivial. 

EigcnTrust [16] is a reputation system developed for P2P networks. Eigen- 
Trust uses transitivity of trust and aggregates trust from peers by performing 
a distributed calculation to determine the eigenvector of a "trust matrix" over 
peers. It relies on some pre-trusted peers, which are supposed to be trusted 
by all peers. EigenTrust (and some other reputation systems like j32[ [33]) is 
designed based on Distributed Hash Tables (DHT) [27, 26, 2] thus imposing sys- 
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tern design complexity and drawbacks, deployment and message overheads [5]. 
[25] proposed a trust system that is built on top of a peer group infrastructure 
using an EigenTrust-like calculation approach. The groups are formed based on 
particular interest criterions and members must follow the set of rules of the 
group they belong to. The authors assumed that a group leader creates the 
group and controls the membership. To calculate trust, the authors introduced 
Eigen Group Trust, which is an aggregative version of EigenTrust. In Eigen 
Group trust, all the transactions rely on the group leaders, who are assumed to 
be trusted and resourceful. 

GossipTrust [31] is another reputation management system designed for P2P 
networks. More specifically, it is designed for unstructured P2P networks. The 
basic idea of GossipTrust is to aggregate global reputation scores using a gossip 
based algorithm. The gossiping process stops when a gossiping error threshold 
is reached. Such approach has the advantage of a fast dissemination of global 
scores with time complexity of 0(log2(n)), where n is the number of peers in 
the network. 

The Beta Reputation System (BRS) [H] is a probabilistic trust model, which 
is based on the beta distribution. In BRS, after each interaction, service re- 
quester gives rating to performance of the service provider, where rating is a 
binary value: positive or negative. The ratings are then used to estimate shape 
parameters that determine the reputation of the assessed agent. However, BRS 
does not show how it is able to cope with false information, which may influence 
accuracy of prediction seriously. 

TRAVOS [30] is a trust and reputation model for agent-based virtual or- 
ganizations. This work also makes use of beta distribution to compute trust 
but it pays more attention to the issue of unfair feedbacks. When a buyer 
evaluates a potential interaction partner (i.e., seller), it first uses its own local 
knowledge to derive trustworthiness of the seller. It then estimates accuracy 
of the direct trust, that is the probability that the real likelihood of cheating 
falls in the certain range from the buyer's estimation. If the direct trust is 
accurate enough, the buyer simply relies on this direct experience based trust. 
Otherwise, it requests feedbacks from other buyers who have interacted with 
the potential seller. To ensure that only accurate feedbacks are considered, 
TRAVOS addresses inaccurate reputation feedbacks by performing two tasks: 
(1) Estimating the probability that a feedback provider's opinion of the seller is 
accurate. This is done by comparing current feedback with the previous feed- 
backs provided by the same buyer. The accuracy of the current feedback is 
the expected value of the beta PDF representing the numbers of successful and 
unsuccessful interactions between the buyer and the seller when the buyer is 
guided by the previous feedbacks. (2) Adjusting reputation feedback according 
to its accuracy to reduce the effect of inaccurate feedback. 

Different from works mentioned above, StereoTrust [T7] uses another kind of 
information, i.e., stereotypes to estimate the initial trust of an unknown agent. 
A stereotype is determined by a feature vector, which can be taken from the 
profile of the agent. To build stereotypes, the agents that trustor has interacted 
with are classified into groups according to the identified features. Stereotypes 



2G 



on each group are calculated by aggregating trustor's past experience with mem- 
bers of that group. Then, when facing the target agent, the trustor estimates 
its trust using stereotypes on groups to which the target agent belongs. Ad- 
ditionally, when some information about stranger's past behavior is available, 
enhanced StereoTrust model (called d-StereoTrust) uses it to refine the stereo- 
type matching. StereoTrust simply aggregates stereotypes by assigning intuitive 
weights to derive trust of the target agent, thus is not able to tell which stereo- 
types are more important than other ones. Differently, by applying sophisticated 
machine learning algorithms, our approach is capable of distinguishing which 
feature is more discriminating thus making the prediction more accurate. 

Similar to StereoTrust, |5] proposed to bootstrap trust of an unknown agent 
through stereotypes, which are estimated based on M5 model tree learning al- 
gorithm [23]. This work also discussed how to combine initial trust (i.e., based 
on stereotypes) and reputation based trust using subjective logic [12] . Differ- 
ently, our approach focuses on transaction instead of individual agent. This 
helps address the inconsistency of the agent behavior since its target is not 
individual agents but overall viewpoint on a group of agents that share the sim- 
ilar properties. Moveover, our approach emphasizes the useful feature selection 
by performing machine learning algorithms and demonstrates its efficacy in a 
realistic setting using a real auction dataset. 

8 Conclusion 

In this paper, we propose a generic trust framework, which is designed for large- 
scale, open systems. Our approach is generically based on machine learning 
algorithms, which provide classification of a potential transaction by learning 
a set of past transactions. In this work we instantiate the framework using 
two common machine learning algorithms, i.e., linear discriminant analysis and 
decision tree. Unlike many existing trust mechanisms, which rely on specific 
agent's historical information to predict its future behavior, our approach only 
uses trustor's local knowledge. This makes our approach quite suitable for a 
large-scale distributed environment where historical information of the agent in 
question is scarce or even not available, and third party information about the 
agent may not be reliable or expensive to obtain. 

In case that trustor does not have sufficient local knowledge, we propose 
to construct a local knowledge sharing overlay network (LKSON) to exchange 
agents' local knowledge (i.e., intermediate results of classification algorithms) 
to predict trustworthiness of a potential transaction. Compared to traditional 
feedback aggregation based trust and reputation systems, such mechanism helps 
(1) reduce the possibility of sharing fake information; (2) avoid privacy leakage, 
(3) save a lot of computation and (4) avoid high communication overhead. 

Not surprisingly, simulation results show that performance of our approach 
is positively correlated with the discrimination power of features on successful 
interactions and unsuccessful interactions. However, compared to other trust 
models, our approach is quite efficient, especially when third party information 
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is not reliable. Moreover, performance of our approach is quite stable because it 
only relies on trustor's local knowledge. The real Allegro dataset based simula- 
tion shows that the proposed approach can be applied to real application (e.g., 
detecting Internet auction frauds). 

In the future works, we are going to investigate more sophistical statistical 
tools (e.g., Multiple Discriminant Analysis, etc.) to further improve accuracy 
of our approach. Another direction is to apply the proposal to some concrete 
applications such as recommending interesting contents in web 2.0 sites (e.g., 
YouTubH Dig£^, etc.). 
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